VIRTUAL HOLOGRAPHIC INPUT METHOD AND DEVICE 
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Background Of The Invention 

[0001] The present invention is directed generally and in various embodiments to an 
apparatus and method for inputting data by interaction with a template display in space. Non- 
holographic versions of remotely sensed human movement and position data input devices 
already exist. For example, Patent Number 6,281,878, Apparatus and Method for Inputting Data 
which is incorporated herein by reference. The present invention in various embodiments goes 
beyond this previous realization by using the actual projected image as the medium for sensing 
the human motion and location as well as a method for allowing 3 dimensional, "floating" 
holographic images to be used. 

[0002] Input devices are used in almost every aspect of everyday life, including 
computer keyboards and mice, automatic teller machines, control panels, vehicle controls, and 
countless other applications. Input devices, like most things, typically have a number of moving 
parts. A conventional keyboard, for example, has moveable keys that open and close electrical 
contacts. Moving parts, unfortunately, are likely to break or malfunction before other 
components, particularly solid state devices. Such malfunction or breakage is even more likely 
to occur in conditions that are dirty or dusty. Furthermore, input devices have become a limiting 
factor in the size of small electronic devices, such as laptop computers and personal organizers. 
For example, to be efficient a keyboard input device must have keys that are spaced at least as far 



apart as the size of the user's finger tips. Such a large keyboard has become a limiting factor as 
electronic devices become smaller. 

[0003] Some prior art devices have attempted to solve one or more of the above- 
mentioned problems. For example, touch screens can sense a user touching an image on the 
monitor. Such devices, however, typically require sensors and other devices in, on, or around the 
monitor. Furthermore, reducing the size of such an input device is limited to the size of the 
monitor. 

[0004] Other prior art devices sense the position of a user's finger using light sensors. 
Those devices, however, often require light sensors to be located above and perpendicular to the 
keyboard, or other input device. As a result, they tend to be bulky and are not suited for use in 
small, hand-held devices. 

[0005] Other prior art devices sense position of a user's finger with light sensors 
located on the surface to be monitored. In the case of a keyboard, for example, such devices 
typically require that sensors be located at the comers or other boundaries of the keyboard. As a 
result, they are bulky because the sensors must be spread out to be at least the same size as the 
keyboard. Such a device does not lend itself to use in a small, hand held device or in providing a 
full sized keyboard, or other input device. 

[0006] All of the abovementioned devices require sensor(s) to operate separate from 
the display (template) for sensing and inputting the data and are relative to a limited two 
dimensional space. 



Summary Of The Invention 

[0007] The present invention in various embodiments incorporates the display and 
sensor as a set of complementary devices and overcomes deficiencies in the prior art by 
providing for an input device that is compact and that allows for a full sized keyboard or other 
input means to be provided. Unlike prior art devices that require sensors to be located directly 
above the area to be sensed or at the boundaries of the area to be sensed, the present invention in 
various embodiments allows the input device to be self contained and remote from the area to be 
sensed and usable in three dimensional space. 

[0008] Other advantages and benefits of the present invention will become apparent 
from the description herein below. 

Brief Description Of The Drawings 

[0009] For the present invention to be clearly understood and readily practiced, the 
present invention will be described in conjunction with the following figures, wherein: 

[0010] FIG. 1 is a diagram illustrating an input device constructed in accordance with 
an embodiment of the present invention. 

[0011] FIG. 2 is a diagram showing the image as a keyboard input device with the user 
interacting with the system. 

[0012] FIG. 3 is a schematic representation the images received and created at the 
sensor and emitter, respectively of the system as well as the imagery transforms of these 
combinations. 



Detailed Description 

[0013] It is to be understood that the figures and descriptions of the present invention 
have been simplified to illustrate elements that are relevant for a clear understanding of the 
present invention, while eliminating, for purposes of clarity, other elements. Those of ordinary 
skill in the art will recognize that other elements may be desirable and/or required in order to 
implement the present invention. However, because such elements are well known in the art, and 
because they do not facilitate a better understanding of the present invention, a discussion of such 
elements is not provided herein. 

[0014] First, embodiments of the invention generate a real time, computer generated, 
digital hologram. A holographic image is generated by shining coherent light on or through a 
piece of photographic film that has been exposed in a specific way and contains an interference 
pattern of the image to be recreated. The coherent light is then redirected (by reflection and/or 
refraction) to reform the original image that it was exposed to. Inasmuch, if the photographic 
film is replaced with a solid state display device 8 (either reflective or Transmissive, LCD type or 
other), and generate the interference pattern via computer generated graphics, the hologram 
image can be created without ever having recorded the original object. Such is a three 
dimensional paradigm of what commonly exists as 2 dimensional computer graphics. 

[0015] Second, the invention utilizes the ability to dynamically record holograms 
digitally, in real time. This is accomplished by setting up a holographic recording scenario 
utilizing a split beam laser to generate a reference beam 12 and an object beam 10. Instead of the 
interference occurring on a piece of photographic film, it occurs at the surface of a two 



dimensional sensor, such as a high resolution CCD video sensor 13. In this setup, the video 
sensor 13 will connect to an analysis engine 17 that performs the image generation and can store 
and/or analyze the incoming interference pattern. This is analogous to what is now used for 
computer image analysis in two dimensions. 

[0016] As shown in figure 1, a hologram of the item to be displayed is generated by 
the output imager 8. For example, a keyboard can be generated. The interference pattern 
required for the object to be generated is created by the analysis engine 17, or learned and stored 
in the analysis engine 17 by showing the sensor 13 the item in holographic recording mode. The 
generated image is seen by the solid state sensor 13. That is, the item is "generated" by the 
display 8, and "seen" by the sensor 13. The interference pattern as seen by the sensor 13 
(generated by the imager 8) is the same nominal pattern as was used to generate the image. An 
analysis engine 17 will compare these images (the pattern used to create the image vs. the actual 
sensed pattern of the image) and find no significant difference. The comparison can be done in 
feature space (analysis of the transform of the pattern into image components) or as a global 
comparison (bit mapping one pattern against the other). The analysis engine 17 can be 
microprocessor based, a hard wired circuit, or any of a number of tools used for this type of 
image or pattern analysis. 

[0017] When someone (or something) interferes with the display as shown in figure 2, 
e.g., in this keyboard scenario the user places a finger on a key 14, the pattern on the sensor 
changes. This new pattern is then processed by the analysis engine 17 to analyze what has 
happened. This can be done by training the differences in the pattern (empirically by training and 



tabulating the results) depending upon where a finger is placed, or by actual transformation and 
analysis of the pattern for an interpretation of what and where the change is. 

[0018] Figure 3 shows schematically an example of what the interference patterns 
representing a keyboard and the interaction of a user might look like. The resulting difference 
pattern 15, which is the result of the elimination of the original keyboard template 9, from the 
newly created pattern 9 and 14, is then transformed into the resulting image 16. 

[0019] In addition, the template image 9 can be modified to respond to the user's 
interactions. For example, in the case of the keyboard, the keyboard image can be reconfigured 
such that the key that the user "pressed" will now be changed in the image to represent a pressed 
position. A "virtual steering wheel" might move with the user turning the wheel to steer. Any 
number of interaction scenarios can be conceived and implemented. 

[0020] Those of ordinary skill in the art will recognize that many modifications and 
variations of the present invention may be implemented. For example, the invention was 
described with respect to a user's finger 14 being used to select items on the input template 9, 
although other things, such as pencils and pens, may be used to select items on the input template 
9. The foregoing description and the following claims are intended to cover all such 
modifications and variations. 

[0021] Figure 4 is a flow chart for the method for detecting an object in an area. The 
method comprises the steps detailed in the chart. 



